Use of the Analysis of Variance Method for Model Order Selection of N-FIR Models

Identification of non-linear FIR-models is studied. In particular the selection of model structure, i.e., to find the contributing input time lags, has been examined. A common method, exhaustive search among models with all possible combinations of the input time lags, has some undesired drawbacks, as a tendency that the minimization algorithm gets stuck in local minima and heavy computations. To avoid these drawbacks we need to know the model structure prior to identifying a model. In this report we show that a statistical method, the multivariate analysis of variance, is a good alternative to exhaustive search in the identification of the structure of non-linear FIR-models. We can reduce the risks of getting an erroneous model structure due to the non-convexity of the minimization problems, reduce the computation time needed and also get a good estimate of how far we can enhance the fit of the desired model. 1 Problem Description Assume that a non-linear FIR model describes the measurements yt from a system with input ut, that is, yt = g(ut, ut−T , ut−2T , ..., ut−kT ) + et. (1) The value of k is unknown in addition to which time lags of ut that contributes to the value of yt and g is an unknown static non-linear function of up to k+ 1 variables. A common method used to estimate the function g is to parameterize it as a non-linear neural network. This method has some drawbacks: Since the model order is not known in advance, it is necessary to assume a largest k and estimate a network of each combination of the time lags of ut under that assumption. Of these models, the one with the best performance on validation data is chosen. Each model takes time to estimate and depending on the assumption of k, there can be many models. During the estimation of the neural network, the optimization algorithm can stop in a local minimum, which can lead to a model with wrong assumptions on relevant input time lags coming out best. Both these drawbacks can be avoided if the structure of the model is known in advance, i.e., the contributing time lags and k are known. In order to achieve